BI-POMDP: Bounded, Incremental, Partially-Observable Markov-Model Planning

Given the problem of planning actions for situations with uncertainty about the action outcomes, Markov models can eeectively model this uncertainty and ooer optimal actions. When the information about the world state is itself uncertain, partially observable Markov models are an appropriate extension to the basic Markov model. However , nding optimal actions for partially observable Markov models is a computationally diicult problem that in practice borders on intractabil-ity. Approximate or heuristic approaches, on the other hand, lose any guarantee of optimality or even any indication of how far from optimal they might be. In this paper, we present an incremental, search-based approximation for partially observable Markov models. The search is based on an incremen-tal AND-OR search, using heuristic functions based on the underlying Markov model, which is more easily solved. In addition, the search provides a bound on the possible error of the approximation. We illustrate the method with results on problems taken from the related literature.